1. Field of the Invention
The present invention relates to data processing apparatuses and methods, learning apparatuses and methods, and media, and in particular, to a data processing apparatus and method for increasing processing efficiency when image data, etc., are processed, and to a medium provided with the same.
2. Description of the Related Art
The assignee of the present invention proposed classification adaptive processing as processing for increasing image quality, etc., and for improving other image features.
The classification adaptive processing consists of classifying processing and adaptive processing. In the classifying processing, data are classified based on their properties, and adaptive processing on the data in each class is performed. The adaptive processing is the following technique.
In the adaptive processing, by linearly linking, for example, pixels (hereinafter referred to as xe2x80x9cinput pixelsxe2x80x9d) constituting an input image (an image to be processed by the classification adaptive processing) and predetermined prediction coefficients, predicted values of pixels of an original image (e.g., an image including no noise, an image free from blurring, etc.) are found, whereby an image in which noise included in the input image is eliminated, an image in which blurring generated in the input image is reduced, etc., can be obtained.
Accordingly, it is, for example, assumed that the original image is teacher data, and it is also assumed that an image obtained by superimposing noise on or blurring the original image is student data. The finding of predicted values E[y] of pixel levels y of pixels constituting the original image by using a linear first-order linking model defined by linear linking of a set of a plurality of student data (pixel levels) x1, x2, etc., and predetermined prediction coefficients w1, w2, etc., is considered. In this case, the predicted values E[y] can be represented by the following expression:
E[y]=w1x1+w2x2+ . . . xe2x80x83xe2x80x83(1)
For generalizing expression (1), by using the following expressions:       X    =          [                                                  x              11                                                          x              12                                            …                                              x                              1                ⁢                n                                                                                        x              21                                                          x              22                                            …                                              x                              2                ⁢                n                                                                          …                                …                                …                                …                                                              x              m1                                                          x              m2                                            …                                              x              mn                                          ]                  W      =              [                                                            w                1                                                                                        w                2                                                                        …                                                                          w                n                                                    ]              ,                  Y        xe2x80x2            =              [                                                            E                ⁡                                  [                                      y                    1                                    ]                                                                                                        E                ⁡                                  [                                      y                    2                                    ]                                                                                        …                                                                          E                ⁡                                  [                                      y                    m                                    ]                                                                    ]            
to define a matrix W composed of a set of prediction coefficients w, a matrix X composed of a set of student data, and a matrix Yxe2x80x2 composed of a set of predicted values E[y], the following observation equation holds:
XW=Yxe2x80x2xe2x80x83xe2x80x83(2)
Here, a component xij of the matrix X represents the j-th student data in an i-th set of student data (a set of student data for use in prediction of the i-th teacher data yi), and a component wj of the matrix W represents a prediction coefficient in which the product of the coefficient and the j-th student data in the set of student data is calculated. A component yj represents the j-th teacher data, and E[yj] accordingly represents a predicted value of the j-th teacher data.
The finding of each predicted value E[y] close to each pixel level y of the original pixel by applying a least square operation to the observation equation (2) is considered. In this case, by using the following expressions:       E    =          [                                                  e              1                                                                          e              2                                                            …                                                              e              m                                          ]        ,            and      ⁢              xe2x80x83            ⁢      Y        =          [                                                  y              1                                                                          y              2                                                            …                                                              y              m                                          ]      
to define a matrix Y composed of a set of actual pixel levels y of the original pixels which are used as teacher data, and a matrix E composed of a set of residuals e of predicted values E[y] from the pixel levels y of the original pixels, the following residual equation holds from expression (2):
XW=Y+Exe2x80x83xe2x80x83(3)
In this case, the prediction coefficients w for the predicted values E[y] close to the pixel levels of the original pixels can be found by minimizing the following squared error:       ∑          i      =      1        m    ⁢      xe2x80x83    ⁢      e    i    2  
Therefore, when the result of differentiating the squared errors with respect to the prediction coefficient wi is zero, the prediction coefficient wi, which satisfies the following expression, is an optimal value for finding the predicted values E[y] close to the pixel levels y of the original pixels.                                                         e              1                        ⁢                          xe2x80x83                        ⁢                                          ∂                                  e                  1                                                            ∂                                  w                  i                                                              +                                    e              2                        ⁢                          xe2x80x83                        ⁢                                          ∂                                  e                  2                                                            ∂                                  w                  i                                                              +          …          +                                    e              m                        ⁢                          xe2x80x83                        ⁢                                          ∂                                  e                  m                                                            ∂                                  w                  i                                                                    =                  0          ⁢                      xe2x80x83                    ⁢                      (                                          i                =                1                            ,              2              ,              …              ⁢                              xe2x80x83                            ,              n                        )                                              (        4        )            
Accordingly, by using the prediction coefficient wi to differentiate expression (3), the following expressions hold:                                                         ∂                              e                i                                                    ∂                              w                1                                              =                      x            i1                          ,                                            ∂                              e                i                                                    ∂                              w                2                                              =                      x            i2                          ,        …        ⁢                  xe2x80x83                ,                                            ∂                              e                i                                                    ∂                              w                n                                              =                                    x                              i                ⁢                                  xe2x80x83                                ⁢                n                                      ⁢                          xe2x80x83                        ⁢                          (                                                i                  =                  1                                ,                2                ,                …                ⁢                                  xe2x80x83                                ,                m                            )                                                          (        5        )            
From expressions (4) and (5), the following expressions are obtained:                                                         ∑                              i                =                1                            m                        ⁢                          xe2x80x83                        ⁢                                          e                i                            ⁢                              xe2x80x83                            ⁢                              x                i1                                              =          0                ,                                            ∑                              i                =                1                            m                        ⁢                          xe2x80x83                        ⁢                                          e                i                            ⁢                              xe2x80x83                            ⁢                              x                i2                                              =          0                ,        …        ⁢                  xe2x80x83                ,                                            ∑                              i                =                1                            m                        ⁢                          xe2x80x83                        ⁢                                          e                i                            ⁢                              xe2x80x83                            ⁢                              x                                  i                  ⁢                                      xe2x80x83                                    ⁢                  n                                                              =          0                                    (        6        )            
When relationships among the student data x, the prediction coefficients w, the teacher data y, and the residuals e, are taken into consideration, the following normalization equations can be obtained from expression (6).                                                                                                                                     (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                            i1                                                    ⁢                                                      xe2x80x83                                                    ⁢                                                      x                            i1                                                                                              )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      1                                                        +                                                            (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                            i1                                                    ⁢                                                      xe2x80x83                                                    ⁢                                                      x                            i2                                                                                              )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      2                                                        +                  …                  +                                                            (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                            i1                                                    ⁢                                                      xe2x80x83                                                    ⁢                                                      x                                                          i                              ⁢                                                              xe2x80x83                                                            ⁢                              n                                                                                                                          )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      n                                                                      =                                  (                                                            ∑                                              i                        =                        1                                            m                                        ⁢                                          xe2x80x83                                        ⁢                                                                  x                        i1                                            ⁢                                              xe2x80x83                                            ⁢                                              y                        i                                                                              )                                                                                                                                                                    (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                            i2                                                    ⁢                                                      xe2x80x83                                                    ⁢                                                      x                            i1                                                                                              )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      1                                                        +                                                            (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                            i2                                                    ⁢                                                      xe2x80x83                                                    ⁢                                                      x                            i2                                                                                              )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      2                                                        +                  …                  +                                                            (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                            i2                                                    ⁢                                                      xe2x80x83                                                    ⁢                                                      x                                                          i                              ⁢                                                              xe2x80x83                                                            ⁢                              n                                                                                                                          )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      n                                                                      =                                  (                                                            ∑                                              i                        =                        1                                            m                                        ⁢                                          xe2x80x83                                        ⁢                                                                  x                        i2                                            ⁢                                              xe2x80x83                                            ⁢                                              y                        i                                                                              )                                                                                                                                                                    (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                                                          i                              ⁢                                                              xe2x80x83                                                            ⁢                              n                                                                                ⁢                                                      xe2x80x83                                                    ⁢                                                      x                            i1                                                                                              )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      1                                                        +                                                            (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                                                          i                              ⁢                                                              xe2x80x83                                                            ⁢                              n                                                                                ⁢                                                      xe2x80x83                                                    ⁢                                                      x                            i2                                                                                              )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      2                                                        +                  …                  +                                                            (                                                                        ∑                                                      i                            =                            1                                                    m                                                ⁢                                                  xe2x80x83                                                ⁢                                                                              x                                                          i                              ⁢                                                              xe2x80x83                                                            ⁢                              n                                                                                ⁢                                                      xe2x80x83                                                    ⁢                                                      x                                                          i                              ⁢                                                              xe2x80x83                                                            ⁢                              n                                                                                                                          )                                        ⁢                                          xe2x80x83                                        ⁢                                          w                      n                                                                      =                                  (                                                            ∑                                              i                        =                        1                                            m                                        ⁢                                          xe2x80x83                                        ⁢                                                                  x                                                  i                          ⁢                                                      xe2x80x83                                                    ⁢                          n                                                                    ⁢                                              xe2x80x83                                            ⁢                                              y                        i                                                                              )                                                                    }                            (        7        )            
By preparing a certain number of student data x and a certain number of teacher data y, the normalization equations (7) can be formed corresponding to the number of prediction coefficients w. Accordingly, by solving equations (7) (although the solution of equations (7) requires a matrix composed of coefficients on the prediction coefficients w to be regular), the optimal prediction coefficients w can be found. For solving equations (7), Gauss-Jordan elimination, or the like, may be used.
The adaptive processing is the above-described processing in which the predicted values E[y] close to the pixel levels y of the original pixels are found based on expression (1), using the prediction coefficients w found beforehand.
The adaptive processing differs from, for example, simple interpolation in that components which are not included in an input image but which are included in the original image are reproduced. In other words, as long as attention is paid to only expression (1), the adaptive processing is the same as interpolation using a so-called interpolating filter. However, in the adaptive processing, the prediction coefficients w corresponding to tap coefficients of the interpolating filter are obtained by a type of learning using the teacher data y for each class. Thus, components included in the original image can be reproduced. That is, an image having a high signal-to-noise ratio can be easily obtained. From this feature, it may be said that the adaptive processing has an image creating (resolution creating) operation. Therefore, in addition to the case where predicted values of an image obtained by eliminating noise and blurring in the original image are found, the adaptive processing may be used for, for example, the conversion of an image having a low or standard resolution into an image having a high resolution.
As described above, in the classification adaptive processing, the adaptive processing is performed for each class. In classification performed before the adaptive processing, a plurality of input pixels are extracted which are adjacent to each of original pixels (hereinafter referred to also as xe2x80x9coriginal pixels of interestxe2x80x9d) based on which a predicted value is found, and the original pixels of interest are classified based on the characteristics of the input pixels (e.g., patterns of the input pixels, an inclination of the pixel levels, etc.). Input pixels fixedly positioned with respect to the original pixels of interest are extracted as the plurality of input pixels for the classification.
Nevertheless, in cases where the classification adaptive processing is used to convert, for example, a blurred input image into a blur-reduced image, the use of input pixels fixedly positioned to original pixels of interest to classify the original pixels of interest regardless of the degree of the blurring in the input image may make it difficult to perform classification that sufficiently reflects the characteristics of the original pixels of interest.
By way of example, when classification adaptive processing on an input image having a small degree of blurring is performed, by preferably performing, in an aspect of image correlation, classification using input pixels relatively close to the original pixels of interest, classification that reflects the characteristics of the original pixels of interest can be performed. When classification adaptive processing on an input image having a large degree of blurring is performed, by preferably performing, in an aspect of adverse effects of the blurring, classification using input pixels relatively far from the original pixels of interest, classification that reflects the characteristics of the original pixels of interest can be performed.
Accordingly, the use of the input pixels fixedly positioned to the original pixels of interest for classification of the original pixels of interest may hinder classification of the original pixels of interest, which reflects their characteristics. As a result, the performance of the classification adaptive processing may deteriorate, that is, an image (here an image obtained by sufficiently reducing blur) obtained by using the classification to sufficiently improve the input image may not be obtained.
In the adaptive processing, input pixels fixedly positioned to pixels of interest are used to perform computation based on the linear prediction expressions (1), whereby predicted values based on pixels of interest are found. Also in this case, it is expected, similarly to the case of the classification that, by preferably performing computation based on the linear prediction expressions (1) with input pixels variably positioned to the pixels of interest, as required, predicted values having small prediction errors from the pixels of interest can be found.
Accordingly, it is an object of the present invention to provide a data processing apparatus and method that solves the foregoing problems, thereby increasing processing efficiency when image data, etc., are processed, and a medium provided with the same
To this end, according to an aspect of the present invention, the foregoing object is achieved through provision of a data processing apparatus for predicting output data corresponding to input data by processing the input data. The data processing apparatus includes a determining unit capable of determining, in accordance with a plurality of data to be extracted from the input data, irregular intervals among the plurality of data to be extracted from the input data, an extracting unit for extracting the plurality of data corresponding to output data of interest to be predicted from the input data in accordance with the determination result by the determining unit, and a predicting unit for finding predicted values of the output data of interest based on the plurality of data extracted by the extracting unit.
According to another aspect of the present invention, the foregoing object is achieved through provision of a learning apparatus for predicting output data corresponding to input data by processing the input data. The learning apparatus includes a determining unit capable of determining, in accordance with a plurality of data to be extracted from the input data, irregular intervals among the plurality of data to be extracted from the input data, and an extracting unit for extracting the plurality of data corresponding to output data of interest to be predicted from the input data in accordance with the determination result, and a computing unit for finding prediction coefficients based on the extracted data.
According to a further aspect of the present invention, the foregoing object is achieved through provision of a data processing method for predicting output data corresponding to input data by processing the input data. The data processing method includes the steps of enabling determination of irregular intervals among a plurality of data to be extracted from the input data in accordance with the plurality of data to be extracted, extracting the plurality of data corresponding to output data of interest to be predicted from the input data in accordance with the determination result, and finding predicted values of the output data of interest based on the extracted plurality of data.
According to a still further aspect of the present invention, the foregoing object is achieved through provision of a learning method for predicting output data corresponding to input data by processing the input data. The learning method includes the steps of enabling determination of irregular intervals among a plurality of data to be extracted from the input data in accordance with the plurality of data to be extracted, extracting the plurality of data corresponding to output data of interest to be predicted from the input data in accordance with the determination result, and finding prediction coefficients based on the extracted plurality of data.
According to yet another aspect of the present invention, the foregoing object is achieved through provision of a storage medium containing a computer-controllable program for predicting output data corresponding to input data by processing the input data. The computer-controllable program includes the steps of enabling determination of irregular intervals among a plurality of data to be extracted from the input data in accordance with the plurality of data to be extracted, extracting the plurality of data corresponding to output data of interest to be predicted from the input data in accordance with the determination result, and finding predicted values of the output data of interest based on the extracted plurality of data.
According to yet another aspect of the present invention, the foregoing object is achieved through provision of a storage medium containing a computer-controllable program for predicting output data corresponding to input data by processing the input data. The computer-controllable program includes the steps of enabling determination of irregular intervals among a plurality of data to be extracted from the input data in accordance with the plurality of data to be extracted, extracting the plurality of data corresponding to output data of interest to be predicted from the input data in accordance with the determination result, and finding prediction coefficients based on the extracted plurality of data.